QUICKEST SEARCH OVER BROWNIAN CHANNELS 



ERHAN BAYRAKTAR AND ROSS KRAVITZ 

Abstract. In this paper we resolve an open problem proposed by [9]. Consider a sequence of 
Brownian Motions with unknown drift equal to one or zero, which may be observed one at a time. 
We give a procedure for finding, as quickly as possible, a process which is a Brownian Motion with 
nonzero drift. This original quickest search problem, in which the nitration itself is dependent on the 
observation strategy, is reduced to a single nitration impulse control and optimal stopping problem, 
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which is in turn reduced to an optimal stopping problem for a reflected diffusion, which can be 
explicitly solved. 



1. Introduction 

qC . 

p | In the classical sequential analysis problem, one observes an i.i.d. sequence of random variables 

{X\,X2, . . .} which all either have distribution Qq or Q\. In deciding whether Qq or Qi is present, 

the agent seeks to minimize some combination of the probability of choosing the wrong distribution 

and the total expected observation time. The first results in this field are due to Wald in |20j . 

who solves the above problem. The problem was first addressed in continuous time by Shiryaev in 
cry . 

' |18| . In one continuous version of the sequential analysis problem, the agent observes a Brownian 

Motion with an unknown drift, which satisfies one of two hypotheses. He seeks to minimize some 
" combination of the probability of misidentifying the drift and the total expected observation time. 
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For a comprehensive introduction to the topic, the reader is referred to [13]. [12] also has an 
excellent chapter on the subject. 

Since their original formulation, many variants of these sequential analysis problems have been 
studied. The problem that we study is known as a quickest search problem. In this class of 
problems, the single channel of information available in the classical problem is replaced with some 
collection of channels, which may be finite or infinite. By observing only one channel at a time, 



the goal is to find a channel satisfying some hypothesis, all the while keeping observation costs 
to a minimum. This type of formulation is a natural one for many physical applications: due to 
hardware or bandwith constraints, our ability to monitor a system may often be constrained to a 
single part at a time. We refer to [9] for a further description of possible applications. 

In [9], the following problem is studied: consider countably many sequences {Y£ : A; = 1,2,...}, 
i = 1, .... For each i, {Y£ : k = 1,2,...} are i.i.d. random variables which obey one of two 
hypotheses: under Hq, Y£ ~ Qq, k = 1, 2, . . ., and under Hi, Y£ ~ Qi, k = 1, 2, . . ., where Qo and 
Qi are two distinct, but equivalent, distributions. At each discrete time k, one can take one of three 
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actions: stop sampling and choose a sequence which is believed to satisfy Hi, continue observation of 
the same channel, or continue observation in a new channel. Over all possible observation strategies 
and their associated stopping times r, our goal is to minimize P{H St = Hq) + cJ5[r] 3 where here 
H St is the true condition of the channel observed at time r, and c represents the cost of making 
one observation. 

The authors of [9] solve the problem above, in the sense that they find an optimal observation 
strategy and stopping time, both of which can be computed as hitting times of an underlying 
posterior process. In this same paper, the authors ask for a solution to the corresponding problem 
in continuous time, and it is this task which we take up. In the quickest search problem in continuous 
time, the basic object of study is a sequence of observable processes Q = 6 l t + Bl, ief}, where each 
B l is an independent Brownian Motion and each 6 % is an independent Bernoulli random variable. 
For some prior tt £ (0, 1) and each i, P^(6 l = 0) = tt, and P%(6 1 = 1) = 1 — n. We say that £ l 
satisfies hypothesis Hq if l = and satisfies hypothesis Hi if 6 l = 1. The general objective 
in the quickest search problem is to find, as quickly as possible, a process which satisfies Hi. 
Observing any process for t units of time incurs a cost of ct, where c > is a constant. One may 
observe only one £ l at a time, but one can instantaneously change between observed processes at 
any time. In discrete time, the problem may be approximated by a finite horizon version solvable 
by backwards induction, but this method is not available in continuous time. Instead, the problem 
is solved by formulating it as a free boundary ordinary differential equation. 

In the literature, there has been a large amount of research into quickest search problems, al- 
though the majority of it has been in discrete time. Since the focus of our paper is a continuous 
time problem, we refer the reader to the references in [9] for an excellent description of the discrete- 
time literature, as well as [ST] for a multi-channel quickest detection problem in discrete time. The 
continuous time literature is sparser, but there are several papers addressing a problem similar to 
ours. The three papers [22] , |14j . and [S] all consider the quickest search variant in which there 
are a finite number of channels, and exactly one channel satisfies H%. For comparison, in the prob- 
lem we study, there are infinitely many channels and no knowledge of how many channels satisfy 
H\. Also in continuous time, in [5], the authors study a multi-channel problem where all channels 
are observed simultaneously: here the goal is to find the common intensity rate of a collection of 
Poisson processes. In [2], two Poisson channels are observed simultaneously to determine the one 
in which disruption occurs first. Other problems involving multiple stopping times include [3] and 
[3], although our problem is closer in spirit to a multiple switching problem than one of multiple 
stopping. 

The technical details of continuous time formulations are somewhat subtle, and indeed, in the 
three papers closest to ours, [22], |14j . and [8] , each purports to fix an error in the previous one. 
For example, in [22J, an optimal switching strategy is described in the following way: consider N 
diffusions 7™, 1 < n < N, which are supposed to represent the posterior for each channel, and 
take the strategy that when 7^ is largest among the N processes, observe channel i. The inherent 
problem in such a strategy is that when, for example, 7^ = 7^ , the sign of 7* — 7s will oscillate 
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infinitely many times when s is in a neighborhood of t. So, such a strategy would necessarily 
switch between channels i and j infinitely many times in that neighborhod, and this is physically 
unfeasible. 

Indeed, even in the discrete time case, certain technical details are not completely developed. To 
rigorously describe the set of all observation strategies in our problem, one must talk of different 
nitrations, since by choosing which channel to observe at different times, we are modulating the 
exact information which we receive. Therefore, one of the aims of this paper is describe a continuous 
time quickest search problem in a mathematically rigorous way. At the same time, however, we can 
obtain a closed form analytic solution to our quickest search problem, both for the value function 
and the optimal stopping time, and so from a practical perspective, our results may be useful in 
the discrete time case if calculating the solution to that problem is too expensive. The theory of 
extended weak convergence, in pQ, provides a possible way to translate insights from continuous 
time optimal stopping back to discrete time. 

The outline of this paper is as follows: we will first describe how problems in continuous time 
sequential analysis can be formulated as optimal stopping problems, using the building block of 
the one channel problem. Much of this material is derived from parts of [12J, Chapter 6, and |19j . 
Chapter 4. We will then state an optimal stopping/switching problem which models our quickest 
detection problem. Its basic feature is that it involves many different nitrations, corresponding 
to the different ways in which one may observe the processes. We show how this problem may 
be reduced to an impulse control/stopping problem with a single filtration and a single Brownian 
Motion, so that the tools of optimal stopping may be applied. 

Let us describe this impulse control/stopping problem more closely, since here the exact structure 
of the problem is evident. Consider a sequence of channels which all have prior probability ft of 
satisfying H\. By observing one channel at the time, the posterior process of precisely one of these 
channels will evolve in time according to a stochastic differential equation, and all the rest will 
simply stay at ft. The goal of the agent is essentially to find some channel and some time when 
the respective posterior process is very close to 1, ensuring that H% is almost certainly satisfied. 
Given this goal, the agent, when faced with a posterior which has dropped below ft, will want to 
immediately move on to the next channel. The agent must take a finite time to react, so after 
some amount of time e, or more realistically, when the posterior process hits the level ft — e, he 
switches channels, and the effect of this is that the posterior process is reset to ft. These are the 
impulse controls available to the agent: at any time, he can move "his" posterior process back to 
ft. Now, in the limit, the agent ideally wants to react as quickly as possible, which means e \, 0. In 
this limit, we show that these impulsed processes converge in an appropriate sense to a diffusion 
with reflecting boundary at ft, so that the original problem may be stated as an optimal stopping 
problem on this reflected process. 

Finally, we solve the optimal stopping problem using a standard verification theorem. Using 
the results derived thus far, we outline e-optimal algorithms for the quickest search problem, and 
provide some computations of optimal threshold levels for different parameters. 
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2. From Sequential Analysis to Optimal Stopping 

Let (£l,J-, P-k) be a probability space supporting a Brownian Motion B and an independent 
Bernoulli random variable 9, with P^{6 = 1) = 7r. We consider two statistical hypotheses 

#1 : 6 = 1 and Pf : 9 = 0, 
which have respective prior probabilities tt and 1 — jr. We let t& = 6t + Bt model the observed 
process, with induced filtration F^ = (jfj ■ We will find it useful to write P% = frPi + (1 — tt)Po, 
where under Pi, £t is a Brownian Motion with unit drift, and under Po it is a standard Brownian 
Motion with zero drift. 

Based on the continuous observation of £, our goal is to choose a sequential decision rule (r, d), 
where r is a stopping time, and d is -measurable and equal to one or zero. Taking d = 1 
models accepting Hi at time r, and taking d = models accepting Hq. The goal is to minimize 
the risk function 

(1) V"(tt) = inf E v [t + al {d=0:9=1} + 61 {d=i e=0} ] , 

(T,d) 

where a and b are used to weight the importance of each misidentification. Let 7rj = P%{9 = ) 
be the posterior process, which models our belief that H\ is satisfied, based on observations up to 
time t, J-f. Given a stopping time r, the choice of whether to set d = or d = 1 is completely 
described by the value of tt t . For c = if ir T < c, d = 0, and d = 1 otherwise. Therefore, (P) 
may be restated as an optimal stopping problem on %t\ 

V(tt) = inf E w [t + avr r A 6(1 - vr T )] . 

T 

We have reduced the sequential analysis problem to one of optimal stopping, but it remains to 
understand the time dynamics of the posterior process ir t . To that end, we introduce the odds 
process 

= a dPi i \ 

* - if „<">■ 

By an application of Bayes' rule, Q t can be calculated, and is equal to exp{£ t — |}. Since the 
posterior process ir t satisfies 

dP-i 



we can relate ixt to We have 



1 + T^r^t 

1 — 7T t 



Now $t clearly satisfies d&t = $td£t by Ito's Lemma. We have a formula for m in terms of &t, so 
using Ito's Lemma again, we derive 



dTTt = -(VT 4 ) 2 (1 - TT t )dt + 7T t (l - 7T t )d£ 



with initial condition ttq = tt. Consider the process Wt = £t — Jo E^[6\J^]ds = it — J^sds. This 
process is immediately seen to be a martingale, and it has continuous paths by construction. A 
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short calculation shows that its quadratic variance is (W)t = t, and so Levy's Theorem implies that 
W is a Brownian Motion. 

We may therefore rewrite ([2]) as 

d-K t = 7T t (l - TT t )dW t , 7T = TT. 

The optimal stopping problem on this diffusion will play a central role in our analysis. 

3. Reduction to a problem with a single filtration 

Let fP, i € N, denote C[0, oo). We consider a sequence of independent Brownian Motions 
W 1 , W 2 , . . ., defined canonically as the coordinate process on J7 N = ft 1 x ft 2 x • • • : for (cj 1 , ui 2 , . . .) € 
r> N , W^oj 1 ,u 2 , . . .) = Let F be the filtration generated by the canonical coordinate process. 
Let P N denote the product measure of Wiener measures on fi N . 

We define, for each i, and for any random time 4> independent of W l , the process ir 1 '^ satisfying 

(3) di$* = tt^(1 - i$+)dW}, for t > (/), ttJ* = vr. 

The value represents the posterior probability, based on observing the history of Channel 
i from time up until time t, that Channel i satisfies Hi. Although <ft here is arbitrary, it will 
always be, for our purposes, a switching time between two channels. We will take these stochastic 
differential equations as our main object of study. Now, given that the single channel sequential 
analysis problem involves the optimal stopping of a single copy of the SDE, it stands to reason 
that a multi channel problem should involve a sequence of these processes which are concatenated 
together according to the way in which we observe each channel. We describe this procedure now. 

Let be the filtration on N generated by W 1 , which coincides with the filtration generated 
by tt 1 . We let be the set of F^-stopping times. 

Let 6 be the set of admissible switching controls, which will determine the set of times when 
the currently observed channel is changed. Elements of 6 will consist of sequences of increasing 
random times {fa, fa, ■ ■ ■} with (j>i = 0. The time fa, i > 2, may be interpreted as the time when 
observation of Channel % — 1 stops and observation of Channel i begins. The main property that 
each <pi should have is that it should be measurable with respect to the information gathered before 
it: our decision to switch should be based on what we have seen thus far. In order to make this 
precise, we will define elements of (3 inductively: given the first n switching times (f>i, fa, ■ ■ ■ , <j>ni 
we will define an allowed (n + l) st switching time. First, the base case. We let <pi = 0. 

The first possible switching time fa is any strictly positive F^-stopping time. So, given fa, we 
define the process tt^'^ 2 as follows: 

The process tt^'^ 2 generates a filtration F^ 2 )'^ 2 . We let T^'^ 2 denote the set of F^'^-stopping 
times. 

Now, we define what the switching time fa may look like, given that fa has already been chosen. 
Such a switching time is any fa £ T^' 4 ' 2 such that fa > fa. Given fa and fa, we define the process 
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^(3)^2^3 as follows: 



M,<h,<l>3 _ _(2),02-, _|_~- 3 >*M 

r t -^t i {i<0 3 }+ 7r < i {*>^ 3 }- 



The process tt®'* 2 '* 3 generates a filtration and T^ 2 '^ 3 denotes the set of wW'**'* 3 - 

stopping times. Proceeding in this way, we define, for each n G N, 7r(™)>^ 2 '"->^ n , F( Tl )' <? ' >2 ' ,,, '^ n , and 
j'{n),<t>2,— ,<f>n_ These are, respectively, the posterior process, filtration, and stopping times which 
result from switching channels at times fa, fa, ■ ■ ■ , fa- 

Definition 3.1. Let $ = {fa, fa, • • •} be a sequence of random times such that (pi > fa-i on the 
set {<j>i-i < oo}, and such that limifa = oo. We say that $ is an admissible switching control if 
fa = 0, fa G T (1) , and for n > 2, fa G f{n-l),<h,~,4>n-x 

We denote by 6 the set of all possible switching strategies <£. Each $ = {fa, fa, ■ • •} G 6 induces 
an observed posterior process 7r*, defined as follows: 

(4) vrf = ^)>^,-,^ Qn the get {0 n < t < 

Intuitively, in comparison with ([3|), 7rf represents the posterior probability that at time t, the 
channel currently being observed under the observation strategy $ satisfies hypothesis H%. If the 
same channel was always observed, 7r* would behave exactly like -k 1 '^ =q . As it is, when the channel 
is switched, the effect on the posterior is a sudden jump back to the original level n. 

The process 7r* induces the filtration along with 7~*, the set of F*-stopping times. We define 
the value function as follows: 

(5) V % = inf V? = inf inf E[cr + (1 - tt*)1. 

$ee $e6 r eT* 

Note that in the value function there is only a (1 — irt) term, instead of irt A (1 — irt). This 
reflects the fact that instead of deciding whether a single channel satisfies H\ or Hq, one is looking 
only for a channel which satisfies H\. Now, the $ also induces a process VF*, which is defined as 
follows on f2 N : 



(6) W t *(w\ w 2 , . . .) = c^ 1 on the set {t < fa}, 
and for n G N 

(7) WjW, . . .) 4 Wj^V.w 2 , . . .) + (a# - W £J on the set {fa < t < fa +1 }. 
We prove the following standard fact: 

Lemma 3.2. For each G &, is a Brownian Motion. 

Proof. We first prove that W* is a martingale. Note that W* can be written as 

oo 

8=1 
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Additionally, 

oo oo 

E[{W?f] = £ E[(W^ i+1 - W^f] = Yi(E[t A <& +1 ] - E[t A &]) = t, 
i=l i=l 

since lim^ = oo. Therefore, by the Dominated Convergence Theorem, 

oo 

r<$> 



E[W^\T S \ = E 



i=l 



= - w^) 

i=l 

= w*, 

the second inequality following from Optional Sampling and the fact that each W % is a martingale. 
Since (W l )t = t for all t, a.s., it follows that (W*)t = i for all i, a.s. It is also clear by construction 
that has continuous paths. Thus by Levy's characterization of Brownian Motion, is a 
Brownian Motion for each $ € (3. □ 

The process vr* has continuous paths, with the exception of jump times at 4>2,4>3, ■ ■ ■■ 
Lemma 3.3. vrf = vr + jj vrf (1 - n*)dW* + - 7r*_)l {t > M 

Proo/. By g]) and ©, on {<f; n < t < <j) n +i}, vrf = vr| n + vrf (l - vrf) dW™. Furthermore, on 
{4>n < t < <f> n +l}, ~ W^ n = WJ 1 - W^ n by construction of . Using the locality of stochastic 
integration (see [15) . the Corollary on p. 62), this implies that 7rf = vr*^ + f, vrf (1 — irf)dW®. In 
particular, vr* n+i _ = vr* + f^ +1 vrf (1 - nf)dW*. Finally, Att* 4 vrf - vr£_ = vr - tt» _. Then, 
on {0 n < t < 4> n +i}, 

vrf = vr* + l\*(l-^)dWf 

= vr* _ + Avrf n + f\f(l-Kf)dW* 

f<t>n ft 

= + / (1 - *?)eWf + vr* - vr* _ + / vrf (1 - vrf )<fiV? 

J 4>n-\ J <t>n 

= + ^ vrf (1 - vrf + vr - vr* n _ 

Now, if we repeated apply this procedure, reducing the index n by one each time, we obtain 
vrf + / vrf(l-7rf)dWf + J> -<_)=* + / vrf(l-7rf)dWf+£(vr-7r*_). 

□ 

Let be another copy of the canonical space C[0, oo) with coordinate process Wt and filtration 
F generated by W. Let P denote Wiener measure on this space. Also, let T denote the set of 
F-stopping times. We would like to reduce the original problem V% to one where everything uses 
the same Brownian Motion iW) and same filtration (F). 
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Lemma 3.4. Let $ £ 6. For any F* 'stopping time r, there exists a T G T such that W® T and 
W ■ Ar are identically distributed as processes. Conversely, for any r G T, there exists a F* -stopping 
time t such that W^ T and W. A r are identically distributed as processes. 

Proof. Let $ G G , and let r G T* . We have a mapping : il N — > Q which is defined according to 
© and ([7]). Since r G T*, it is in particular measurable with respect to the filtration F* generated 
by W* on Q N . This implies that for any tJ G 0, t is constant on (W®)~ 1 (lj). 

Thus, we define f : !] 1 as follows. For uJ € O, choose any oj G (W®)~~ 1 (oJ), and set 
t(uJ) = t(uj). From the discussion in the above paragraph, this definition is well-defined. 

We claim that r is an F-stopping time. For t G M, we have 

{Id : t(57) < = {cJ : r(w) < t for w G (VF*)" 1 ^)} 
= ({w : t(uj) < t}) . 

Since r G T*, the set {uj : t(uj) < t} G J-**. By construction, the mapping : f2 N — > takes 
J-"*-measurable sets into J-Vmeasurable sets. Therefore, r G T. 

Next, we claim that W* T and VF.at are distributed identically as processes. As before, this is 
essentially a tautology. Let A G .Poo- 

We have 

P ■ W. A7m (uJ) G A} = {uJ : W* T(w) (w) G j4 for a; G (W^)- 1 ^)} 
= ({ W :< (u) ( W )eA}). 

Thus, P({ZJ : W. AT (B) G A}) = P (V* ({a; : W^ t(w ) (w) G ^}))- Since is a Brownian Mo- 
tion, the measure P* which induces on Q agrees with P. Thus, P({lo : W. A t(zj) 6 A}) = 
P* (V* ({a; : W* tH (w) G A})) = P N ({a; : W* t(w) (o;) G A}). Thus, W. AT and W* T are iden- 
tically distributed. 

Conversely, suppose that r is an F-stopping time. Define r : S1 N — > R by r = r o VF*. We claim 
that r is a stopping time. Let t G M. Then 

{w : t(w) < = {lu : t(W*(lu)) < t] 

= (W*)- 1 ({cJ : t{W) < t}) . 

Since r is an F-stopping time, the set {ZJ : t(uJ) < t} G Pt, and (VF*) _1 ({ZJ : r(uJ) < t}) G J 7 *. 
So, r G T*. 

Now we claim that W^ T and T^.at are identically distributed as processes. Let A G P°°. 
{uj : W\ t[lo) G A} = {uj : W. AT ^(uJ) G A for uj such that = 57} 

= (^r 1 ({ZJ:W. A ^)(5J)GA}). 

As before, 

^ N ({" : e ^}) = P N ((VF*)" 1 ({or : F. A ^)(a;) G A})) 

= P* ({57:W. Ar(s) (57)€A}) 
= P[{u:W. AT(p) (u)eA}), 
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and so the processes are identically distributed. □ 

Lemma 3.5. For each $ £ 6, there exists a sequence of F-stopping times <& = {cfii = 0, fa,...} 
such that for 

(8) Tff 4 7T + / Wf (1 - Tff )dW s + - Tff Jl {t> ^ } , 

7f* is identically distributed with 7r*. 

Proof. Let iV* be the simple point process on Q N which jumps at the F*-stopping times <pi, (j>2, 

Let $ = , </> 2 , . . .} be a sequence of F-stopping times whose existence is guaranteed by Lemma 

13.41 and let N be the simple point process on Q which jumps at the F-stopping times <f>^ <^ 2! 

According to Lemma 13.41 (W*,iV*) and (W, N) are identically distributed as processes. Let 
f(x) = x(l — x) and let g(x) = tt — x. Then 7r* and 7r* satisfy the SDE's 



d4 = f«)dWr + g(^t)dN i 



t ! 



and 



dwt = f(wt)dW t +g(wt)dN t 



Note that if 7T* starts inside the interval (0, 1), then it stays there for all time, and similarly for 
7f*. On the interval (0, 1), f(x) is bounded and Lipschitz, and the same goes for g(x). By Theorem 
9.1 of [B] (see p. 245 — 6), the above SDE's have uniqueness in law. Consequently, 7r* and 7f* are 
identically distributed. □ 

The converse is proven similarly using Lemma 13.41 

Lemma 3.6. Let $ = {(fti, (/>2, • • •} be a collection of F-stopping times which increase to infinity, 

and let <I> induce 7f as in Then there exists f £ 6 such that 7r* is identically distributed to 
— $ 

7T . 

Lemma 3.7. We have 

(9) Va = jnf_ inf£[cr + (1 - tt*)] , 

where & is the set of all sequences $ = {<f>i, </>2> • • •} of F-stopping times which increase to infinity. 

Proof. Denote by V# the right side of ©. Let $ 6 @, and consider the optimal stopping problem 
inf E[ct + (1 — 7r*)]. By Lemma 1331 the process irf is distributed identically to 7f*. According 

t€T* 

to Lemma 2.3 of [10], the value function associated to the optimal stopping of a process depends 

only on that process 's distribution. Therefore inf E[ct + (1 - nf)] = inf E[ct + (1 - Iff)] > V*. 

rer* reT 

Taking the infimum over all $ € 6, we obtain 

v* > v*. 
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Now, let $6 6. By Lemma 13, 6\ there exists <I> € & such that 7r* is identically distributed with 
. So, using the same reasoning as above and taking the infimum over all we obtain 

□ 

4. Working with the new problem, and reduction to an optimal stopping problem 

Prom now on, we will drop the overline notation, and simply write <S, $,7Tt, W, F, T, for, re- 
spectively, the set of allowed switching strategies, an arbitrary switching strategy, the posterior 
process, the single Brownian Motion W, the filtration induced by W, and the stopping times for 
that filtration. 

Let 7T° denote the posterior process when there is no switching. In other words, tt° satisfies 
the SDE dir® = 7r^(l — n®)dWt along with 7r° = tc. We next define the reflected process tt t with 
boundary at tt: 

(10) dirl = vr[(l - n r t )dW t + dA t , 

where At is continuous, non-decreasing, flat off of 7r r = n, Aq = 0. 
We also have an optimal stopping problem associated with 7r r : 

(11) VT= inf £[cr+(l-<)] 
Lemma 4.1. VI < 14. 

Proof. Let <3? = {<fii,4>2, • ••}£©• Fix i; we will show that 7r|" > 7r* on [0j,</>, + i). By construction, 
7r?. = 7r, and on the interval <fo+i), the dynamics of 7r* are described by the diffusion dnf = 
7r*(l — 7r*)c2Wt. Let 7r 0,< ^ be the un-switched diffusion starting from 7r^. = tt, so that Tr ' 9 ^ = 7r* on 
</>i+i). Note that, by construction, if,, > n. Then (|10p and the comparison theorem for SDE's 
(i.e. Theorem 54 p. 324 of [TB]) imply that tt t > 7r 0,< ^ on [<pi,4>i+i), and so ir r > 7r* on 
It now follows that ir r t > 7r* for all i, a.s. Consequently, for any r € T, E[(l — -ir r T )] < 1£[(1 — vr* )], 
implying that VT < V^-. □ 

Remark 4.2. T/ie comparison theorem (see |15] ) /or SDE's essentially says that if X 1 and X 2 are 
solutions to SDE's with identical volatility terms and the drift term for X 1 is greater than or equal 
to the drift term for X 2 , then X 1 will be greater than or equal to X 2 . 

Following |17j . p. 146, we give the Skorokhod representation of 7r r . Given a process Y and i S M, 
the Skorokhod representation consists in finding a process X and an increasing process A such that 
X = Y + A, X > 7r, and f£°(X s — n)dA s = 0, i.e. A only increases when X = tt. 

Let a(x) = (1 — x)x, and let Y solve the SDE 

Y t ^TT+ [ a{Y s + A s (Y))dW s , 
Jo 
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MY)= sup {(y s -7r)-}. 

0<s<t 

As in [IT], the SDE does in fact have a unique strong solution. Then, if we set X t = Y t + A t (Y), it 
is clear that ic r = X. 

Let e > 0. We outline a parametrized family of switching strategies (impulse controls). Let 
<& e denote the strategy that switches channels whenever the observed posterior process hits the 
level fc — e. induces the process it 6 , starting from tcq = ft, which diffuses according to dit\ = 
7r|(l — TTf)dWt on (ft — e, 1). When it reaches the level ft — e, it is instantaneously brought back to 
ft (i.e. switched). We wish to give a Skorokhod type representation of it e . Consider the SDE 

r t 

rt A 



(12) Y?±ft + I a(Y* + A\{Y'))dW s 

where 



- sup {(17-vr)-} 

£0<s<t 



Note that A e s (-) is not even continuous with respect to the uniform norm on continuous paths. 
Therefore, the standard theory does not imply that the SDE (|12|) has a strong solution. We can, 
however, show that a solution exists by a piecewise construction. 

Lemma 4.3. For each e > 0, the SDE (|12j) has a strong solution. Moreover, for X^ = Y t e + Af(Y e ) , 
X e = vr e . 

Proof. Consider the SDE 

(13) Y ? = Y? 1 ±*+ fa(Y^)dW s . 

Jo 

As er(-) is Lipschitz and bounded on the interval (0, 1), it is known (see Theorem 11.5 of [T7]) that 
(I13j) has a strong solution. Let r e '° = 0, and r 6 ' 1 = inf{t > : Yf' 1 = ft — e}. Note that on the 
random time interval [0, t 6 ' 1 ), Y 6 ' 1 solves the SDE (|12p . For t > r e ' , consider next the SDE 



(14) 



n> 2 ±Y^ 1+ J^a(Y s ^ + e)dW s . 



As before, (1141) has a strong solution. Let t 6 ' 2 = inf{t > t 6 ' 1 : Y^' 2 = ft — 2e}. Then on [r 6 ' 1 , t 6,2 ), 
Y e ' 2 solves (|12p . Arguing inductively, we define i^ e ' n , for t > r e,n_1 , by 

(15) Y? n 4 y^-i + a(y/>" + (n - l)e)dW a , 

which has a strong solution as before, and the stopping time r e,n = inf{t > r e,?1_1 : Y e ' n = fc — ne}. 
Defining the process Y e by Y t e = Yf ,n for t € [r e ' n_1 , r e '™), n > 1, it is apparent that Y e solves the 
SDE (H2J). 

For the last claim, note that when A e (Y € ) is constant, dXf = a(Y t e + Af(Y e ))dW t = a(Xl)dW t . 
The times when A € (Y e ) jumps (by e) correspond to the impulses from ft — e to ft. □ 

Lemma 4.4. For any t,e > 0, we have E [(Y — Y 6 )^ 2 ] < 8te 32t e 2 . In particular, for any t > 0, 
(Y - Y e )* t ->0 inL 2 ase->0. 
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Proof. Let K be the Lipschitz constant of cr(-) on (0, 1). Write 
E(Y e - Y)f = E 



E 



< E 



(* + J q °( Y s + A e s (Y e ))dW s -7T- J^a{Y s + A s {Y))dW s 
(a(Y e + A e (Y e )) ■ W - a(Y e + A.{Y e )) ■ W + a(Y e + A.(Y e )) -W-a(Y. + A.{Y)) ■ W^j 
a{Y e + A e {Y e )) ■ W - a(Y e + A.{Y e )) ■ W)* 



+ (a(Y e + A.(Y e )) ■ W - a(Y. + A.(Y)) ■ W)* t 



< 2E 



, *2 



, *2 



( {a(Y e + A e (Y e )) - a(Y e + A.(Y e ))) ■ W)\ 
+ 2E [( (a(Y e + A.{Y £ )) - a(Y. + A.(Y))) ■ W)* f ' 
= (l) + (2), 

with the second inequality above following from (a + 6) 2 < 2a 2 + 26 2 . Next, using the Burkholder- 
Davis-Gundy Theorem for the first inequality and the iT-Lipschitzness of cr(-) for the second, 

(1) < 2C 2 E f {o-{Y: + A e B (Y e )) - a(Y: + A S (Y*))) 2 ds 

Jo 

< 2C 2 K 2 E I\y* + A e s {Y e ) - Y s e - A s (Y e )) 2 ds 

Jo 

< 2C 2 K 2 e 2 t; 

the last inequality follows from the fact that for a given path uj, sup |^4 s (w) — ^4|(w)| < e. The 

0<s<t 

constant C 2 is a universal constant arising from the Burkholder-Davis-Gundy Theorem. The L 2 
version used above actually can be proven using Doob's L 2 -inequality for martingales, and from 
this the explicit formula C 2 = 4 can be derived. For details, see p. 14 of [7]. 

Next, we note that A.(-) is Lipschitz continuous with respect to the uniform norm on continuous 
paths, with Lipschitz constant 1. Applying this fact for the third inequality below, Burkholder- 
Davis-Gundy for the first inequality, the i^-Lipschitz continuity of a(-) for the second inequality, 
and Fubini's Theorem in the last inequality, we obtain 

(2) < 2C 2 E I (a(Y s e + A s {Y e )) - a(Y s + A S (Y))) 2 ds 

Jo 

< 2C 2 K 2 E I\y* + A s (Y e ) -Y s + A s (Y)) 2 ds 

Jo 

< 8C 2 K 2 E /V/ " Ysfds 

Jo 

< 8C 2 K 2 [ E(Y e - Y)* 2 ds. 

Jo 



For each e > 0, define f e : 



by f € (s) = E(Y e - Y)* s 2 . According to the above 



reasoning, f e (t) < 2C 2 K 2 te 2 + 8C 2 K 2 /* f € (s)ds. By Gronwall's Lemma, it follows then that 
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f e (t) < 2C2K 2 te 2 e 8C2R2t . Since all processes in question live in the interval (0, 1), we may assume 
that K = 1. Therefore, f e (t) < 8te 32t e 2 . In particular, t, (Y e - Y)* t -> in L 2 as e -> 0. □ 

Corollary 4.5. For any t,e > E [(7r e — vr r )j 2 ] < 16te 32t e 2 + e. In particular, as e — > 0, (ir e — 
7r r )t -> in L 2 . 

Proof. Write vr[ = y+A^Y) and ir e t = Y t e +A e t (Y e ). We have shown in LemmaOthat (Y e -Y)* t -> 
in L 2 as e -> 0. Therefore, it suffices to show that (A.(Y) - A € (Y e ))* -> in L 2 as e -> 0. So, for 
any s > 0, 

|A(y) - Al(Y^)\ < \A S (Y) - A S (Y*)\ + \A S (Y*) - A\{Y')\ 
<(Y-Y*)* s + e, 

where we have used the Lipschitz continuity of A s (-) with respect to the uniform norm. Therefore, 
(A.(Y) - A e (Y e ))* t < (Y — Y e )* t + e, which converges to in L 2 as e — s> 0. The quantitative estimate 
is also clear. □ 

Lemma 4.6. VT = V~ 

Proof. In light of Lemma 14.11 it suffices to show that VT > V% . Without loss of generality, we 
assume that VT < oo; otherwise, there is nothing to show. Let {r n : n £ N} be a sequence of 
stopping times such that E[cr n + (1 — 7r£ )] \. VT. Fix 6 > 0, and choose n sufficiently large so that 
£[cT n + (l-<J] <VT + 5. 

Next, we note that the processes tt t and ir e are all bounded, so that in particular, they are uni- 
formly of Class D. Therefore, for a suitably large t, it is the case that E [tt^ l{ Tn>t }] , E \x% n l{ r „>t}] < 
5 for each e > 0. By Corollary 14. 5| for e sufficiently small, \E [7r£ n l{T n <t}] ~~ E [ 7r r n l{r„<<}] | < <5- 
Thus, 

\VT-E [cr n + (1 - <J] | < |V? - £ K + (1 - <J] I + 1^ K + (1 " <J] " E K + (1 - <„)] I 

= |V? - £ [cx„ + (1 - <J] I + |B [<J - E [<J | 

< <^ + \E Kl{r»>t}] " # [< n W*J I + ^ KW*}] - E i<nHr n <t}) 

<5 + 25 + 5 
= 48. 

Since V* < V? e < E [cT n + (1 - <J] , it now follows that V* < VT. □ 

5. Optimal stopping of the reflected diffusion 

We wish to relate the optimal stopping problem VT = inf E[ct + (1 — 7r£)] to an ODE with a 
free boundary. First, we look for / : [77", 1] — > M and it* G (-fr, 1) that satisfy: 

1 d 2 f 

(16) -[x(l-x)] 2 ^ = -c,^<x<vr*, 

(17) /(a:) = l-a;,7r*<x<l, 
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(18) /' (v r)=0,/V) = -l. 

Notice in particular that we require / to be C 1 at tt and tt*; tt* must be chosen to ensure that 
this happens. 

Lemma 5.1. The problem (|16p .(fT7|). (|18p has a unique solution, for precisely one tt* G [ft, 1). 

Proof. One may verify directly that the function *&a,b(%) — 2c(l — 2x) log + Ax + B = 'I'(x) + 
Ax + B, for constants A and B, is the general solution of ([16)) . We will show that the boundary 
conditions are satisfied for precisely one vr*, ^4, and 5. 

~2*-2(*-l)ftlog( T E 5F )-l" 



The condition /'(tt) = forces ^4 = — ^'(-fr) = —2c 



(?T— 1)7T 



. Since \I/(x) + Ax is 

-oo (so 

oo), it follows that there is a unique 7r* € (jt, 1) such that ^'(7r*) + A = — 1. Define 



strictly concave and ^'(x) + A is continuous on 1), ^'(tt) + A = 0, and lim ^(x) + Ax 

xfl 

lim ty'(x)+~A -- 
xfl 

B so that B satisfies the equality I - tt* = ^^(tt*) = $>(tt*) + ~Att* +B. Taking /(x) = ^^(x) 
for x G [vr,vr*) and /(x) = 1 — x for x G [7r*, 1] yields the unique solution to (fT6]) . ([TT)) . ([T8|) . □ 

Our candidate for the value function is therefore 



(19) 
with 



A 



2c(l -2x)log T ^ +Ax + B ifO<x<vr* 

1 — X if 7T* < X < 1 



-2c 



27r-2(7r-l)7rlog -1 



TT 



1)tT 



7r* satisfying 2c 



2vr* 



2(7r*-l)7r*log^j-l 
(tt* - 1)tt* 



+ A 



-1, 



B satisfying 1 - vr* = 2c(l - 2tt*) + Att* + B. 



For each x £ [w, 1], we set VT(x) = inf [cr + (1 — 7rl)l, where the expectation E x [-] denotes 
expectation under the probability P x , i.e. P x (tt t q = x) = 1. We now claim that f(x) is equal to 
the value function V.T(x). Consider the set V = {/ G C^([-7r, 1)) : /'(7r) = 0}. The infinitesimal 
generator C r of tt t satisfies, for / G V, C fix) = ^x 2 (l — x)f"(x). 

Lemma 5.2. For f{x) as above, VJ(x) = /(x). 

Proof. We wish to apply a verification theorem for optimal stopping problems, Theorem 10.4.1 of 
[TT] . p. 225. We must check that /(x) defined by ([To]) . ([TT)) . (fT8~)) satisfies the nine hypotheses of that 
theorem. Note that several inequalities are reversed because our problem involves a minimization 
over all stopping times. Let G = [tt, 1], and let D = {x G G : f(x) < 1 — x}. 

(i) / G C 1 (G): This is true by construction. 

(ii) / < 1 — x on G: At tt*, /(vr*) = 1 — tt*. Since f(x) is concave down, /(x) < 1 — x for 
x G [vr,7r*], and by construction f(x) = 1 — x on [tt*, 1]. 
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(iii) E x [J °° lr 7r *T.(7rJ)(fe] = 0: This follows from the fact that the speed measure of it r s is 
m r (dx) = ;^p~p • Now, i.e. Proposition 3.10 of [16J, p. 307 may be applied. 

(iv) dD is Lipschitz: This is trivial in the one-dimensional problem here. 

(v) / € C 2 (G\{tt*}) and the second order derivatives of / are bounded near tt*: For x € (tt, n*), 
f"(x) = x -i^f x y2 , which is bounded on (tt,tt*), and for x G (71"*, 1), f"(x) = 0. 

(vi) C r f + c > on G \ D: For x € G \ D, C r f + c = + c > 0. 

(vii) C r f + c = on L>: For x £ D, C f + c = ±x 2 (l - x) 2 (ji^ja) + c = 0. 

(viii) td = inf{t > : 7r[ ^ £)} < 00, P x -a.s. for each x € G: Using the same argument as in 
(iii), Proposition 3.10 of [16J implies that E x [td] < 00 for each x € G. 

(ix) The family {ir^ : t < te>,t £ T} is P a -uniformly integrable for any x £ G: This is 
immediate, using the fact that ir r is bounded. 

Having checked all the hypotheses of the verification theorem, we deduce that f(x) = VT(x). □ 

6. A ROUGH ALGORITHM FOR QUICKEST SEARCH 

Using the methods of the previous sections, we can describe near optimal algorithms for quickly 
finding a channel which satisfies hypothesis H±. We outline a procedure below for finding an 
e-optimal strategy. 

(1) Fix e > 0. 

(2) For given values of c, ir, calculate the threshold it* = tt*(c,tt) via Lemma 15.11 Let r = 
inf{i > : 7T( = 7r*} be defined for any version of tt t . 

(3) Choose t > sufficiently large so that P(t > t) < |. This can be done, for example, by 
calculating E[t] via the speed measure of ir r . 

(4) Choose 62 sufficiently small so that 16te 32 *e2 + £2 < f • 

(5) Adopt the switching strategy $ €2 , in which the observed channel is switched whenever the 
posterior level hits 7r — €2- 

(6) The switching strategy $ £2 induces the observed Brownian Motion W = W^ 2 . Using W, 
construct the solution to the SDE Y t = vr + J * a(Y s + A s (Y))dW s , and set X t = Y t + A t (Y). 
Let t* = inf{t > : X t = vr*}. 

(7) At time t* , accept hypothesis H\ for the channel which is currently being observed. 

Applying the reasoning of Lemmas 15.11 and 14.61 we may deduce that this observation/stopping 
strategy will be e-optimal. 

7. Numerical Results 

In this section, we illustrate our previous results by computing the optimal threshold level for 
various levels of the observation cost c and prior it. The data below, which can be found in the 
second appendix, is directly calculated from the value function established in Section 4. We first 
plot the threshold levels against the observation cost c, when the prior tt is fixed. As indicated by 
Tables [T]and [2] below, for fixed 7r, vr*(c) decreases with c. This is not surprising, because the higher 
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the running cost for observations, the lower one's standards will be for accepting the hypothesis 
that a channel satisfies H±. 

Next, we plot the threshold levels against the prior n, when the observation cost c is fixed. As 
indicated by Tables [3] and [4] below, for fixed c, ir*(n) increases with tt. Again, this is not surprising. 
The higher the prior belief that all channels satisfy Hi, the more restrictive one should should be 
in selecting a channel believed to satisfy that hypothesis. 

8. Appendix: Tables of data 
Table 1. Optimal thresholds 7r*(c), for tt = .5. 



c 


7T*( C ) 


c 


7T*( C ) 


.0025 


.995 


.05 


.865 


.005 


.989 


.06 


.840 


.01 


.977 


.07 


.815 


.02 


.950 


.08 


.793 


.03 


.922 


.09 


.773 


.04 


.893 


.1 


.755 



Table 2. Optimal thresholds vr*(c), for tt = .75. 



c 


7T*( C ) 


c 


tt*(c) 


.0025 


.995 


.05 


.911 


.005 


.990 


.06 


.899 


.01 


.979 


.07 


.889 


.02 


.959 


.08 


.879 


.03 


.941 


.09 


.871 


.04 


.925 


.1 


.864 



Table 3. Optimal thresholds 7r*(7r), for c = .01 



TT 


tt*{tt) 


TT 


vr*(vr) 


.025 


.704 


.5 


.977 


.05 


.951 


.6 


.978 


.1 


.968 


.7 


.979 


.2 


.973 


.8 


.980 


.3 


.975 


.9 


.982 


.4 


.976 


.95 


.985 
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Table 4. Optimal thresholds 7r*(7r), for c = .03 



7T 


7T ^7r; 


7T 


7T ^7r; 




041 


. <j 


999 


.05 


.164 


.6 


.930 


.1 


.690 


.7 


.937 


.2 


.867 


.8 


.946 


.3 


.898 


.9 


.960 


.4 


.913 


.95 


.972 



References 

[1] David Aldous. Weak convergence and the general theory of processes. Incomplete draft of a monograph. Available 

at http://www.stat.berkeley.edu/ aldous/Papers/weak-gtp.pdf 1981. 
[2] Erhan Bayraktar and H. Vincent Poor. Quickest detection of a minimum of two Poisson disorder times. SIAM 

J. Control Optim., 46(1):308-331 (electronic), 2007. 
[3] Rene Carmona and Savas Dayanik. Optimal multiple stopping of linear diffusions. Math. Oper. Res., 33(2) :446- 

460, 2008. 

[4] Rene Carmona and Nizar Touzi. Optimal multiple stopping and valuation of swing options. Math. Finance, 
18(2):239-268, 2008. 

[5] Savas Dayanik, H. Vincent Poor, and Semih O. Sezer. Sequential multi-hypothesis testing for compound Poisson 

processes. Stochastics, 80(l):19-50, 2008. 
[6] Nobuyuki Ikeda and Shinzo Watanabe. Stochastic differential equations and diffusion processes, volume 24 of 

North-Holland Mathematical Library. North-Holland Publishing Co., Amsterdam, second edition, 1989. 
[7] Ioannis Karatzas and Steven E. Shreve. Brownian motion and stochastic calculus, volume 113 of Graduate Texts 

in Mathematics. Springer- Verlag, New York, second edition, 1991. 
[8] Eugene M. Klimko and James Yackel. Some aspects of search strategies for Wiener processes. In Statistical 

decision theory and related topics (Proc. Sympos., Purdue Univ., Lafayette, Ind., 1970), pages 225-237. Academic 

Press, New York, 1971. 

[9] Lifeng Lai, H. Vincent Poor, Yan Xin, and Georgios Georgiadis. Quickest search over multiple sequences. IEEE 

Trans. Inform. Theory, 57(8):5375-5386, 2011. 
[10] Sabrina Mulinacci and Maurizio Pratelli. Functional convergence of Snell envelopes: applications to American 

options approximations. Finance Stoch., 2(3):311-327, 1998. 
[11] Bernt Oksendal. Stochastic differential equations. Universitext. Springer- Verlag, Berlin, sixth edition, 2003. An 

introduction with applications. 
[12] Goran Peskir and Albert Shiryaev. Optimal stopping and free-boundary problems. Lectures in Mathematics ETH 

Zurich. Birkhauser Verlag, Basel, 2006. 
[13] H. Vincent Poor and Olympia Hadjiliadis. Quickest detection. Cambridge University Press, Cambridge, 2009. 
[14] Edward Posner and Howard Rumsey, Jr. Continuous sequential decision in the presence of a finite number of 

hypotheses. IEEE Trans. Inf. Theor., 12(2):248-255, September 2006. 
[15] Philip E. Protter. Stochastic integration and differential equations, volume 21 of Stochastic Modelling and Applied 

Probability. Springer- Verlag, Berlin, 2005. Second edition. Version 2.1, Corrected third printing. 
[16] Daniel Revuz and Marc Yor. Continuous martingales and Brownian motion, volume 293 of Grundlehren der 

Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer- Verlag, Berlin, 

second edition, 1994. 



18 



[17] Chris Rogers and David Williams. Diffusions, Markov processes, and martingales. Vol. 2. Cambridge Mathemat- 
ical Library. Cambridge University Press, Cambridge, 2000. Ito calculus, Reprint of the second (1994) edition. 

[18] Albert N. Shiryaev. Two problems of sequential analysis. Cybernetics, 3(2):63-69 (1969), 1967. 

[19] Albert N. Shiryaev. Optimal stopping rules, volume 8 of Stochastic Modelling and Applied Probability. Springer- 
Verlag, Berlin, 2008. Translated from the 1976 Russian second edition by A. B. Aries, Reprint of the 1978 
translation. 

[20] Abraham Wald. Sequential tests of statistical hypotheses. Ann. Math. Statistics, 16:117-186, 1945. 
[21] Qing Zhao and Jia Ye. Quickest detection in multiple on-off processes. IEEE Trans. Signal Process., 58(12):5994- 
6006, 2010. 

[22] Kamil S. Zigangirov. On a problem of optimal scanning. Teor. Verojatnost. i Primenen, 11:333-338, 1966. 

(Erhan Bayraktar) Department of Mathematics, University of Michigan, 530 Church Street, Ann 
Arbor, MI 48104, USA 

E-mail address: erhan@umich.edu 

(Ross Kravitz) Department of Mathematics, University of Michigan, 530 Church Street, Ann Arbor, 
MI 48104, USA 

E-mail address: ross.kravitz@gmail.com 



